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An Operational Analysis of System Calibration 



Donald P. Gaver 
Naval Postgraduate School 
Hasan B. Mutlu 
AT&T Bell Laboratories 



Mathematical models are proposed for studying the impact of mis-calibration 
upon operational effectiveness. Methodology for assessing the system effective- 
ness and an approach for optimizing the effectiveness of a calibration program 
are examined. The theory application is discussed and the results of some 
specific and convenient models are presented. 



1. Introduction 

The effectiveness of many systems depends upon the degree of the calibration of their subsys- 
tems. For example, a ship with navigational equipment that is out of calibration may not be 
able to locate its destination, or, in the case of a Navy ship, locate an adversary. If a Navy 
ship’s weapon system is also out of calibration the difficulties will be compounded. An analo- 
gous problem arises in connection with engine de-tuning, when fuel consumption likely 
increases and performance decreases, and with drift of communication systems. Similar rea- 
soning also applies to hardware and/or software configurations of computer 
systems, e.g. in factory robotic systems where mechanical re-calibrations 
occur and are controlled by software systems. The detrimental effect of 
mis-calibration is well recognized; the systems are taken to ranges or 
other facilities for testing and recalibration. 
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The purpose of this paper is to develop simple mathematical models to investigate ways of 
dealing with mis-calibration. If the various aspects of the problem can be assembled, some gui- 
dance is then available to deal with it effectively. Although many realistic elements of the 
problem can be considered, the fundamental issue is this; given that important subsystems 
depart from calibration and effectiveness as time passes, it is desirable to determine a schedule 
for re-calibration to optimize system operational effectiveness. Frequent calibration of impor- 
tant systems would be highly desirable if this were a cost-free operation. However, in reality 
the operational cost of calibration is time during which the system is unavailable for, or so 
degraded as to be incapable of adequately performing, its operational purpose. It should be 
mentioned that the actual dollar cost in man-hours, of performing calibration and the degrada- 
tion of the calibration equipment could be considered, but these aspects are ignored as secon- 
dary effects. 

The problem discussed here is recognizably analogous to those formulated and analyzed in 
several other areas. Similar issues arise in industrial quality control; see Lorenzen and Vance 
(1986), who utilize renewal-reward theory, and Smith and Vemuganti (1968), who invoke stan- 
dard Bayesian decision theory. The important area of system availability monitoring, in both 
military and civilian nuclear power industry, requires a similar analysis; Thomas, Jacobs and 
Gaver (1987) recently formulated such problems and solved them using dynamic programming 
techniques. But the present paper is intended to provide a skeletally simple model for a partic- 
ular situation, leaving mathematical elaboration aside. Furthermore, we emphasize operationally 
relevant costs and penalties (i.e., times out of service), and not simple monetary costs, although 
such can be brought in. 

Figure 1 is an idealized graph of operational effectiveness against time. The periods of duration 
C denote those periods during which the system has zero effectiveness because it is undergoing 
calibration and is therefore out of the operational area, and the periods of duration T represent 
those periods during which the system is operational, but diminishing in effectiveness. The 
graph suggests that, if effectiveness drops with time, there will be an optimal value for T, a 
best period, T* , at which to calibrate. We now present how such a period may be determined. 
More complex and realistic models are also introduced. 
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Figure 1 . Idealized Graph of Operational Effectiveness. 



2. Mathematical Models 

For a simple mathematical treatment let e(t) be the effectiveness (e.g., the probability of suc- 
cessful operation), at time t after the calibrated system returns to service. Let C be the time 
required for calibration, and T the duty or on-station time. Then the average effectiveness over 
a cycle of length T + C, and hence in the long run, is 



e(T) = 



f e(t)dt +0 



T + C 



( 2 . 1 ) 



the term 0 represents and emphasizes the total lack of effectiveness during the calibration 
period. In order to maximize e(T) it is useful to study the derivative 
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T 



de(T) 

dJ 



(T + C)e(T)- \e(t)dt 



o 



( 2 . 2 ) 



( T+C ) 2 



as it depends on 7: if de(T)/dT = 0 for 7 > 0 then 7* is a candidate for a time between the 
end of one calibration and the beginning of the next. Equivalently, equation (2.2) asks if there 
is a positive solution 7*, of 



for fixed positive C. The fact that such a solution always exists and that it defines an optimum 
can be established from the usual second derivative criterioa Since the optimal 7 satisfies 
equation (2.3), it turns out that at the optimum the average effectiveness over an entire cycle 
equals the effectiveness at the time the active part of the cycle ends; or symbolically as fol- 
lows: 



where: the over-bar signifies the time average of effectiveness over 7* + C. All of the above 
tacitly assumes that the function e(t ) is regular enough for the mathematical operations 
invoked: e(t) > 0 decreasing and twice-differentiable is more than adequate; if e(t) is a proba- 
bility then e(t) <, 1. 



T 




(2.3) 



?(7*) = e(7*) 



(2.4) 



To increase understanding, we will examine some simple specific models. 



2.1 Linear Effectiveness Loss 
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Let 



Jl-af, 0 S f S a' 1 

jO, a -1 <, t 



(2.5) 



so that the downward-sloping parts of the graph of Figure 1 are strictly linear. Then equation 
(2.3), the equation for optimal T = T* , is as follows: 



1 - aT = — (T - ^7' 2 ), 0 < T < a' 1 . 

T+C 2 



( 2 . 6 ) 



It is clear that no value of T > a 1 can be optimum. The equation (2.6) simplifies to the qua- 
dratic as follows: 



aT 2 + 2 aCT - 2C = 0 



(2.7) 



with a single positive solution 



T* = -C + ^C 2 + 2 C/a 



( 2 . 8 ) 



at which the optimum value of effectiveness is as follows: 



e{T*) 



t' 

J e (t)dt 
_o 

T* +C 



= 1 - aT 
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= (1 + aC ) - Vc 2 + 2 aC 



= (1 + aC) - ^(1 + aC) 2 - 1 (2.9) 



It is interesting that the solution depends only upon the parameter aC, the product of calibra- 
tion drift rate, a, and the length of the re-calibration period, C. For instance, if aC — » 0 then 
effectiveness approaches unity if the rate of calibration degradation, a, approaches zero, or the 
calibration time, C, approaches zero, or both, or one approaches zero more rapidly than the 
other gets large. Alternatively, this shows that equal-effectiveness or a-C tradeoff curves are 
simple hyperbolas in the (a,C) plane. A tendency in this direction may well be common. 

The above model is rather crude, but is easy to understand. It resembles the tool wear problem 
of Smith and Vemuganti (1968) without the Bayesian refinements, and with operationally 
relevant costs. The following model (Linear Degradation with Diffuse Damage) is more quali- 
tatively appealing. 



2.2 Linear Degradation with Diffuse Damage 

Consider a more specific model for effectiveness, one that relates to damage inflicted on a tar- 
get after time t has elapsed, and the system has developed an (unsuspected) bias of magnitude 
at. At this time, the x-y error made in locating a target is assumed to be given by the joint 
Gauss/normal density as follows: 



f(x,y,t ) 



_ L_ P vn 1 Qc-at) 2 1 (y-ar) 2 

2nd 2 2 a 2 2 a 2 



( 2 . 10 ) 



If a cookie-cutter damage function, with radius R, is in effect (0% effective if x 1 + y 2 > R 2 , 
100% effective if x 2 + y 2 <> R 2 ) then: 



e(t)= jj f(x,y\t)dxdy . 
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However, this is difficult to work with, and even overly simplistic. Instead, suppose that a von 
Neumann-Gauss diffuse damage function can be used; (i.e., that the probability of critical dam- 
age to a target located at (0,0) by a weapon with impact point (x,y) is equal to 5(x,y) = exp(- 
a(x 2 + y 2 ))) (Eckler and Burr (1972)). Then the following: 



e(t)= | \Ux,y)f (x,y\t)dxdy 



1 

2ko 2 



J \ zxp[-a(x 2 +y 2 )]f (x,y\t)dxdy 



= ( 



1 



2na ■ 



exp 



1 (at-x? 

2 a 2 



exp[-c vc 2 ]dx? 



( 2 . 11 ) 



by virtue of the symmetry assumed; almost free of charge we can consider asymmetrical dam- 
age functions, but the opportunity is declined. The above integral is evaluated at sight: it is 
seen to be essentially the convolution of two normal densities. After squaring, as demanded by 
equation (2.11), the result is: 



e(t) = 



l/2a 

(o 2 +l/2a) CXP 



(at? 

(ct 2 +l/2a)_ 



( 2 . 12 ) 



Instead of dropping off linearly, as in the previous case, e(t) first diminishes rather slowly, later 
falling quite rapidly (exponentially fast) towards zero: by the time at = M3 2 + l/2a, effective- 
ness e(t) is just below 40% of its maximum, while if ca = 0.5? c? + l/2a, effectiveness e(f) is 
about 78% of the maximum; finally if at = 0.25^o 2 + l/2a, effectiveness e(t) is 94% of the 
maximum. Note that the maximum effectiveness is (1 + 2ao 2 ) -1 ^ 1; if either a 2 or a become 
large, meaning that if either weapon effectiveness falls off rapidly, with miss distance (a large), 
or the ultimate weapon delivery variance is great (a 2 large), then even maximum effectiveness 
is low. 
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In order to solve for the optimum T' write (from equation (2.3)) the following: 



(T+C)exp 



(aT) 2 

(ct 2 +l/2a) 




(a 0 2 

(a 2 +l/2a) 



dt . 



Change the variables to the dimensionless version as follows: 



x = (aT)/(a 2 + l/2<x) u2 ; y=(aC)/(a 2 + l/2a) l/2 , 



so one can solve the following dimensionless equation, once and for all, for x*: 



(x+y)exp (-x?) = j exp {-z 2 )dz ; 
o 



the positive value of x, namely x*. that satisfies this equation may be located by 
Raphson, or even graphically: one can plot, for given y, 



L(x) = (x-fy)exp(-x 2 ) 
and 

X 

R(x) = j exp(-z 2 )dz 

o 



(2.13) 



(2.14) 



(2.15) 



Newton- 



on the same piece of paper, vs. x. 



The arbitrary selected y values and the corresponding x* values from the computer program, 
which solves the dimensionless equation (2.15), are shown in Table 1. 



Y 


0.01 


0.02 


0.03 


0.04 


0.05 


0.06 


0.07 


0.08 


0.09 


X* 


0.236 


0.302 


0.346 


0.381 


0.410 


0.434 


0.456 


0.476 


0.494 



TABLE 1. y and Values. 



In order to redefine x and y in equation (2.14) and the effectiveness formula equation (2.12) in 
a more meaningful form, again change variables to the following: 



v = l/2a ; p = — s — ; k - alo 
cr+v 



(2.16) 



where: v and p might be called vulnerability and probability of success respectively, and k is 
constant. So one can write the following: 



x = kT^l-p ; 



Y = kC^p 



(2.17) 



We focus attention on the representation equation (2.12) in what follows, mainly for analytical 
and computational convenience: 



e(t) = p exp[-(&) 2 (l-p)] . 



(2.18) 



Thus, the preceding expression, at the optimum point T , leads to the following relationship: 
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e(T*) = p expi-ix*) 2 ) 



(2.19) 



and consequently from equation (2.17), the optimal proportion of on-station time can be 
obtained as follows: 



X* _ 7~* 

Y C 

T* = — ?* , (2.20) 

k^X-p 



so the proportion of on-station time is, under optimum conditions, as follows: 



r* 



T* 

T*+Y 



( 2 . 21 ) 



Since the optimum t values are available from Table 1, one can easily cal- 
culate the optimal effectiveness, (2.4) (hereafter called simply effective- 
ness) given some constant variance and V or only p. Some of the results 
are tabulated in Table 2 as an example. Note that the effectiveness de- 
creases as the variance increases while V is held constant. 

Example : Suppose a - 1.5 yds/month, C = 0.5 month, a 2 = 20 (yds) 2 and p = 0.9 arc given. 
First find y from equation (2.17) as 0.053, then look up the corresponding x* value from Table 
1, which is 0.417. From equation (2.20) T* is 3.93 months, from equation (2.21) the propor- 
tion of on-station time is 88.7%, and from equation (2.19) an average effectiveness of 75.6% 
can be obtained. 
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Effectiveness 


(oMO; v=200) 


0.901 


0.869 


0.845 


0.824 


0.805 


0.789 


0.773 


0.759 


(o 2 = 20; v=200) 


0.860 


0.830 


0.806 


0.786 


0.768 


0.753 


0.738 


0.725 



TABLE 2. Effectiveness for Constant Variance (a 2 ) and v. 



2.3 Linear Degradation with Diffuse Damage Using Random Drift 

As an alternative model, that incorporates the possibly different drift rates of different indivi- 
dual systems, suppose that the drift, a, is a random variable with an appropriate distribution 
function instead of a constant as in equation (2.12), namely, the effectiveness conditional on a 
is as follows: 



e ( t \a) 



l/2ot 

(o 2 +l/2a) 



exp 



(a 0 2 

(O 2 +l/20t) 



( 2 . 22 ) 



Then the expected average, or unconditional effectiveness over a cycle of length T + C, in the 
long run, becomes the following: 



T 

jE[e«:q)]dt 

^ (r ^ = ± TTc- 



(2.23) 



In order to be specific, and also so that explicit mathematical results are obtained, let a 2 have a 
gamma distribution function with parameters X and (3, as follows. 
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/ 2 2(x;X,p) 



HP) 



X, p > 0 , 



(2.24) 



and use the following for fixed a 2 , (i.e., the square of the drift rate away from calibration), 



Eia 2 ) = a 2 = ; Var(a 2 ) = . 



(2.25) 



Now use equation (2.22) and (2.23) to obtain the following: 



E[e(t\a)) 



l/2a 

(cr 2 +l/2a) 





a 2 +l/2a 



/ £ 2(x;X,p)d[x . 



After substituting the gamma density function, it is easy to see that the result of the integration 
yields the following: 



E[e(f,a)] 



l/2a 

(o 2 +l/2q) 




IP 






o 2 +l/2q 



(2.26) 



or equivalently, in view of equation (2.25) the following: 



E[e(f,a)) 



l/2q 



(cr 2 +l/2q) 



IP 



1+ 



(at) 2 



(o 2 +l/2q)P 



(2.27) 



Various analytical properties, of the previously described model, will now be recorded. These 
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properties provide useful insights into the behavior of the effectiveness at time t. 
1. If (3 — > oo, equation (2.27) becomes the following: 



E[e(t;q)] = expHar) 2 ) 



which reflects the fact that if P increases, the variance of a in the distribution 
decreases towards zero, and the situation reduces to that of previous model. 

2. If P -» 1, then the following: 



E[e(f,q)) 



i/2ct 

(o 2 +l/2a+(ar) 2 ) ’ 



which is larger than the effectiveness in the equal-drift case. 

In order to solve for the optimum 7*. write the following for the general case: 



T+C 



1 + 



(aT) z 



(ct 2 +l/2a)P _ 



= J 



1+ 



(a tf 



(a 2 +l/2a)p 



-dt 



Change the variables as follows: 



x = (a7)/(o 2 +l/2a) ,/2 ; 7= (aC)/(a 2 +l/2a) 1/2 , 



(2.28) 



of drift rate 



(2.29) 



(2.30) 



(2.31) 



so that one can solve the dimensionless equation as follows: 
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x+y 




V 



X 



s 



dz 




(2.32) 



the positive value of x, namely x*. that satisfies this equation for any constant p may be found 
by a computer program. In fact, one may get the solution for the special case P = 1 by making 
use of arctg integration for the right-hand side as follows: 



Y = ( arctg x) (1 + x 2 ) - x . 



(2.33) 



In general, the right-hand integral can be transformed to the integral of a Student’s t density, 
and the t-tables found in most statistics books can be used to evaluate it. 

The arbitrary selected y values and the corresponding x* values for P = 1 are presented in 
Table 3. 



Y 


0.01 


0.02 


0.03 


0.04 


0.05 


0.06 


0.07 


0.08 


0.09 


X * 


0.239 


0.307 


0.355 


0.392 


0.424 


0.451 


0.476 


0.499 


0.520 



TABLE 3. y and x* Values (p = 1). 



At this point, it is very easy to calculate the effectiveness given some constant variance and v 
or only p from equation (2.19). Some of the results are listed in Table 4 as an example. As in 
the previous case, effectiveness decreases as the variance increases when v is held constant 
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Effectiveness 



(02=20; v=150) 


0.835 


0.806 


0.784 


0.765 


0.748 


0.733 


0.719 


0.706 


(o2=30; v=150) 


0.788 


0.761 


0.740 


0.722 


0.706 


0.692 


0.679 


0.667 



TABLE 4. Effectiveness for Constant Variance (o 2 ) and v ((3 = 1). 



3. Conclusions 

The purpose of this paper has been to show that quite simple mathematical models can provide 
useful ways of studying the impact of mis-calibration upon operational effectiveness. Metho- 
dology for assessing the system effectiveness and an approach for optimizing the effectiveness 
of a calibration program have been examined. We have concentrated here on specific and con- 
venient models, but it is obvious that other more elaborate mathematical models can be treated 
similarly. The relative effectiveness of different system configurations can also be investigated. 

The possibility exists that operational data will reveal different underlying distributions, and 
may suggest alternatives for evaluating effectiveness other than the ones described in this 
paper. This possibility could be profitably investigated. 
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